-
Notifications
You must be signed in to change notification settings - Fork 121
[ABTest] Reload all experiments irrespective of logged in/out context #8788
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ABTest] Reload all experiments irrespective of logged in/out context #8788
Conversation
Generated by 🚫 dangerJS |
You can test the changes from this Pull Request by:
|
itsmeichigo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This works as expected to me 🎉 I like that this solution is simpler and more effective than making refreshUserData async and making changes everywhere.
What I'm concerned about is why we needed to register AB tests by context before. Will this affect the results of logged-in experiments if many users fail to reach the logged-in state?
|
In case starting the logged-in experiments too early can affect their results, how about we take a slightly different route:
WDYT? @selanthiraiyan |
This is a great point. I think you are right that this change will affect the results of experiments without an exposure event. We should explore other options.
By doing this, we might end up using an old expired value as the variation instead of fetching and showing the latest variation value. We might be able to overcome this by using the Time-to-live value. I don't know whether I am missing any other edge cases. |
Considering that Plus, we will always fetch the assignments again after the user relaunches the app, so using cached values right after logging out should work IMO. |
I think you are right. 👍
I agree. I will explore this solution and raise a different PR. Thank you, @itsmeichigo! |
Closes: #8782
Description
Problem
User who was originally assigned
treatmentvariation for a logged-out experimnent sees a different variation for the same experiment after logging out.Reason for the change in assignment
state,AppCoordinatornavigates to the login screen here.controlvariant is shown to the user.My fix in this PR
We fetched the variations for only specific contexts (logged in/out) when analytics is reset. I changed that behaviour and instead started fetching the variations for all available experiments.
anonIDand auth token values to identify the device and user and provide correct variation values.To verify this change works as expected
A better solution (Not in this PR)
Wait for the ABTest variations to reload and proceed.
stateis being set toDeauthenticatedStateeven before we clear/reset other things.WooAnalytics.refreshUserData()method is currently called fromSwitchStoreUseCase.finalizeStoreSelection()and upon switching store inDefaultStoresManager.removeDefaultStore(). I hope one of these calls can be avoided.deauthenticationfromDefaultStoresManager.deauthenticate()synchronizeAccountfromDefaultStoresManager.synchronizeAccount()Testing instructions
treatmentvariation for thewoocommerceios_login_rest_api_project_202301_v2experiment and you can test this PR (If not you need to use a sandbox to for a variation. You can follow the instructions from this PR to assign a varition to your device.)treatmentvariation.Screenshots
NA
RELEASE-NOTES.txtif necessary.